Method for tracking moving object

ABSTRACT

A method for tracking a moving object is provided. The method detects the moving object in a plurality of continuous images so as to obtain space information of the moving object in each of the images. In addition, appearance features of the moving object in each of the images are captured to build an appearance model. Finally, the space information and the appearance model are combined to track a moving path of the moving object in the images. Accordingly, the present invention is able to keep tracking the moving object even if the moving object leaves the monitoring frame and returns again, so as to assist the supervisor in finding abnormal acts and making following reactions.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Taiwan applicationserial no. 97144397, filed on Nov. 17, 2008. The entirety of theabove-mentioned patent application is hereby incorporated by referenceherein and made a part of this specification.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing method. Moreparticularly, the present invention relates to a method for tracking amoving object.

2. Description of Related Art

Visual monitoring technology becomes more important in recent years,especially after the 911 event, more and more monitoring cameras areallocated at various places. However, a conventional surveillance isgenerally carried out by human monitoring, or monitoring data is juststored in a storage device for later reference. As more and more camerasare allocated, the required human labour is increased accordingly, sothat an automatic monitoring system applying a computer visual techniqueplays an important role in recent years.

The visual monitoring system analyses behaviours (such as a movingtrack, a pose or other features) of a moving object in a monitoringframe to detect occurrence of an abnormal event, so as to effectivelynotify securities for handling. A plurality of basic issues of thevisual monitoring, such as background subtraction, moving objectdetecting and tracking, shadow removing, etc. have been studied andresearched in the past. Recently, the study and research are focused ondetection of high-class events, such as behaviour analysis, remnantsdetection, wander detection or congestion detection, etc. For a currentstrong demand of monitoring market, automatic and intelligent behaviouranalysis may have a great demand and business opportunities.

The so-called wander detection refers that one or a plurality of movingobjects is continually and repeatedly appeared in a certain monitoringregion within a specific period of time. For example, prostitutes orbeggars may wander about street corners, graffiti writers may stayaround walls, person having intent to commit suicide may wander about aplatform or drug dealers may wander about a subway station to wait theirclients, etc.

However, since a camera vision of the visual monitoring system islimited, which cannot totally cover a moving path of a wanderer, whenthe wanderer leaves the monitoring region, the visual monitoring systemthen loses the monitoring object, and cannot continue the detection,especially when the wanderer returns again. Therefore, how tore-identify the wanderer to associate his former behaviour is abottleneck for the current wander detection technique.

SUMMARY OF THE INVENTION

Accordingly, the present invention is relates to a method for tracking amoving object, by which space information and appearance model of themoving object in a plurality of images are combined to continually tracka moving path of the moving object in the images.

The present invention provides a method for tracking a moving object.The moving object in a plurality of continuous images is detected, so asto obtain space information of the moving object in each of the images.An appearance feature of the moving object in each of the images iscaptured to build an appearance model of the moving object. The spaceinformation and the appearance model are combined to track a moving pathof the moving object in the images.

In an embodiment of the present invention, the step of detecting themoving object in a plurality of continuous images further includesjudging whether the moving object is a tracking object, and removing themoving objects not belonged to the tracking object. Wherein, a method ofjudging whether the moving object is the tracking object includesjudging whether an area of a rectangular region is greater than a firstpredetermined value, and if the area is greater than the firstpredetermined value, the moving object encircled by the rectangularregion is judged to be the tracking object. Another method is to judgewhether an aspect ratio of the rectangular region is greater than asecond predetermined value, and if the aspect ratio is greater than thesecond predetermined value, the moving object encircled by therectangular region is judged to be the tracking object.

In an embodiment of the present invention, the steps of capturing theappearance features of the moving object in each of the images to buildthe appearance model of the moving object are as follows. Therectangular region is divided into a plurality of blocks. A colordistribution of each of the blocks is captured. A median of the colordistribution is obtained from each of the blocks according to arecursive method to build a binary tree for describing the colordistribution. The color distributions of branches of the binary tree areselected to serve as a feature vector of the appearance model of themoving object.

In an embodiment of the present invention, the step of dividing therectangular region into the blocks includes dividing the rectangularregion into a head block, a body block and a lower limb block accordingto a certain proportion, and the step of capturing the colordistribution of each of the blocks includes neglecting the colordistribution of the head block. Wherein, the color distribution includesa color feature in a RGB color space or in a hue, saturation,illumination (HSI) color space.

In an embodiment of the present invention, after the step of detectingthe moving object in a plurality of continuous images to obtain thespace information of the moving object in each of the images, the methodfurther includes tracking the moving path of the moving object accordingto the space information, and accumulating a elapsed time of the movingobject staying in the images.

In an embodiment of the present invention, after the step ofaccumulating the elapsed time of the moving object staying in theimages, the method further includes judging whether the elapsed time ofthe moving object staying in the images is greater than a firstpredetermined time, wherein if the elapsed time is greater than thefirst predetermined time, the appearance features of the moving objectare captured to built the appearance model of the moving object, and thespace information and the appearance model of the moving object arecombined to track the moving path of the moving object in the images.

In an embodiment of the present invention, the steps of combining thespace information and the appearance model of the moving object to trackthe moving path of the moving object in the images are as follows. Aspatial related prior probability of the corresponding moving object intwo adjacent images is calculated according to the space information,and a similarity of the corresponding moving object in the two adjacentimages is calculated according to the appearance information. The priorprobability and the similarity are combined to a Bayesian tracker tojudge the moving path of the moving object in the adjacent images.

In an embodiment of the present invention, when the elapsed time isgreater than the first predetermined time, the method further includesrecording the elapsed time and the appearance model of the moving objectinto a database, which includes associating the appearance model of themoving object to a plurality of appearance models within the database tojudge whether the appearance model of the moving object is recorded inthe database. Wherein, if the appearance model of the moving object isalready recorded in the database, only the elapsed time of the movingobject is recorded into the database; conversely, if the appearancemodel of the moving object is not recorded in the database, the elapsedtime and the appearance model of the moving object are recorded into thedatabase.

In an embodiment of the present invention, the steps of associating theappearance model of the moving object to the appearance models withinthe database include calculating a first distance between the appearancemodels built at two different time points based on the same movingobject, so as to build a first distance distribution, and calculating asecond distance between the appearance models of two moving objects inthe images, so as to build a second distance distribution, and thenobtaining a boundary of the first distance distribution and the seconddistance distribution to serve as a standard for distinguishing theappearance model.

In an embodiment of the present invention, after the step of recordingthe elapsed time and the appearance model of the moving object into thedatabase, the method further includes analysing a time sequence of themoving object in the database to determine whether the moving object isaccorded to a wander event. A determining method thereof includesjudging whether a time for the moving object continuously appearing inthe images is greater than a second predetermined time, wherein if thetime for the moving object continuously appearing in the images isgreater than the second predetermined time, the moving object is judgedto be accorded to the wander event. Another method is to judge whether atime interval for the moving object leaving the images is less than athird predetermined time, wherein when the time interval for the movingobject leaving the images is less than the third predetermined time, themoving object is judged to be accorded to the wander event.

In the present invention, by building an appearance model of a visitor,and by combining a Bayesian tracking technique, a database managementtechnique and an adaptive threshold learning technique, the movingobject entering a monitoring frame is continuously tracked, so that aproblem that when the moving object leaves the monitoring frame andreturns again, the moving object cannot be continuously detected can beresolved. Moreover, the wander event of the visitor can be automaticallydetected according to a time condition that the visitor appears in themonitoring frame.

In order to make the aforementioned and other objects, features andadvantages of the present invention comprehensible, a preferredembodiment accompanied with figures is described in detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide a furtherunderstanding of the invention, and are incorporated in and constitute apart of this specification. The drawings illustrate embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention.

FIG. 1 is a schematic diagram illustrating a moving object trackingsystem according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a method for tracking a moving objectaccording to an embodiment of the present invention.

FIGS. 3( a), 3(b) and 3(c) are schematic diagrams illustratingappearance models of a moving object according to an embodiment of thepresent invention.

FIG. 4 is a schematic diagram illustrating a binary tree of a colordistribution according to an embodiment of the present invention.

FIG. 5 is a flowchart illustrating a Bayesian object tracking methodaccording to an embodiment of the present invention.

FIG. 6 is a flowchart illustrating a method for managing a visitordatabase according to an embodiment of the present invention.

FIGS. 7( a), 7(b) and 7(c) are schematic diagrams illustrating a methodfor renewing an adaptive threshold according to an embodiment of thepresent invention.

FIG. 8 is a diagram illustrating calculation of an adaptive thresholdaccording to an embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

The present invention provides an unsupervised wander detectiontechnique, by which a system can automatically learn specific parametersaccording to events of a monitoring area, so as to build an appearancemodel for a visitor entering the monitoring area, and associate theappearance model to a visitor database for analysis. Wherein, bycomparing the appearance model to historical records, the system cankeep tracking the visitor even if the visitor leaves the monitoring areaand returns again. Finally, according to a predefined wandering rule, awander event can be further detected. To fully convey the spirit of thepresent invention, embodiments are provided to describe the presentinvention in detail.

FIG. 1 is a schematic diagram illustrating a moving object trackingsystem according to an embodiment of the present invention. Referring toFIG. 1, the tracking system 100 of the present invention detects themoving object in a plurality of continuous images based on a backgroundsubtraction 110. Since a tracking object is a moving object having anintegral appearance (for example, a pedestrian), objects that are notbelonged to the tracking object are removed according to a simplecondition setting, which is referred to as moving object capturing 120.

On the other hand, for each of the captured moving objects, anappearance feature calculation 130 is first performed. Then, a movingobject tracking 140 is continuously performed by a tracker based on theBayesian decision, and an appearance model of the moving object is builtbased on the appearance features of the same moving object obtained froma plurality of the images. Meanwhile, the tracking system 100 maintainsa visitor database 150 in a memory. Wherein, visitor database management160 is performed to compare and associate the appearance features of thecurrent captured moving object to the appearance models in the visitordatabase 150 according to a result of an adaptive threshold renewing170. If the moving object can be associated to a certain object withinthe visitor database 150, it represents that the moving object has everbeen to the scene before; conversely, the moving object is added to thevisitor database 150. Finally, a wander event detection 180 can beperformed according to a time condition that the visitor appears in thearea. When the moving object is tracked, the tracking system 100 takes adistribution status of the moving object in the frame as a sample toautomatically learn how to distinguish differences among differentvisitors, so as to correctly associate the appearance models. Next, adetailed flow of a method for tracking the moving object according to anembodiment of the present invention is provided.

FIG. 2 is a flowchart illustrating a method for tracking a moving objectaccording to an embodiment of the present invention. Referring to FIG.2, in the present embodiment, the moving object entering the monitoringarea is tracked to build the appearance model thereof, and theappearance model is compared to data stored in a visitor database of asystem memory to determine whether the moving object has ever appeared,so as to continuously track the moving object. Detailed description ofthe method is as follows.

First, the moving object in a plurality of continuous images is detectedto obtain space information of the moving object in each of the images(step S210). According to such moving object detection technique, abackground image is first built, and a foreground is obtained bysubtracting the background image from a current image. Each ofconnection regions of the foreground obtained by subtracting thebackground image from the current image is marked according to aconnected component labeling method, and is recorded by a minimumrectangular region b={r_(left), r_(top), r_(right), r_(bottom)} capableof encircling the connection region, wherein r_(left), r_(top),r_(right) and r_(bottom) respectively represent a left, a top, a rightand a bottom boundary of the rectangular region.

It should be noted that since there are plenty of factors that can formthe foreground, and the object of interest is a foreground objectcontaining a single moving object, a pedestrian is taken as an examplein the present invention, and whether the moving object is thepedestrian is further judged, and the moving object not belonged to thepedestrian is filtered, wherein the filtering is performed based on twofollowing conditions. A first condition is to judge whether an area ofthe rectangular region is greater than a first predetermined value, andif the area is greater than the first determined value, the movingobject encircled by the rectangular region is judged to be thepedestrian, by which noises and fragments can be filtered. A secondcondition is to judge whether an aspect ratio of the rectangular regionis greater than a second predetermined value, and if the aspect ratio ofthe rectangular region is greater than the second predetermined value,the moving object encircled by the rectangular region is judged to bethe pedestrian, by which blocks containing overlapped figures or largerange noises can be filtered.

Next, the appearance features of the moving object in each of the imagesare captured to build an appearance model of the moving object (stepS220). In detail, the present invention provides a novel appearancedescription method, by which a meaningful appearance feature is obtainedaccording to a color structure and a relatively loose body division. Theso-called relatively loose body division means that the rectangularregion encircling the pedestrian is divided into a plurality of blocks,and a color distribution of each of the blocks is captured. For example,the rectangular region can be divided into a head block, a body blockand a lower limb block according to a proportion of 2:4:4 forcorresponding to the head, the body and the lower limb of thepedestrian. Wherein, since a color feature of the head is liable to beinfluenced by a facing direction thereof, and a discrimination thereofis not significant, information of the head block can be neglected.

For example, FIGS. 3( a), 3(b) and 3(c) are schematic diagrams ofappearance models of a moving object according to an embodiment of thepresent invention. Wherein, FIG. 3( a) illustrates an image ofpedestrian, which is a rectangular region filtered based on the abovetwo conditions, and is referred to as a pedestrian candidate P, FIG. 3(b) illustrates a corresponding connection object. After the connectionobject is labeled, a median of the color distribution is respectivelyobtained from the body block and the lower limb lock of FIG. 3( c)according to a recursive method to build a binary tree for describingthe color distribution.

FIG. 4 is a schematic diagram illustrating a binary tree of a colordistribution according to an embodiment of the present invention.Referring to FIG. 4, M represents the median of a certain colordistribution of the body block or the lower limb block, and ML and MHare medians of respective color distributions divided from M. Deduced byanalogy, branches MLL, MLH, MHL and MHH are obtained. Wherein, the colordistribution can be the color feature of one of a RGB color space, a HSIcolor space, or other color spaces, which is not limited by the presentinvention. For simplicity's sake, the RGB color space is taken as anexample in the present invention to build a binary tree containing threelayers of the color distribution, by which a feature vector f=[R_(MLL)^(body), G_(MLL) ^(body), B_(MLL) ^(body), R_(MLH) ^(body), . . . ,R_(MHH) ^(legs), G_(MHH) ^(legs), B_(MHH) ^(legs)] with 24 dimensions isformed to describe the appearance feature of the pedestrian. After thefeature vector is obtained, each of the pedestrian candidates is thenrepresented by the space information and appearance model in the image.

After the space information and the appearance model of the movingobject are obtained, the two information are further combined to track amoving path of the moving object in the images (step S230). In thepresent embodiment, tracking of the moving object is achieved accordingto a moving object tracking method based on the Bayesian decision.According to such tracking method, the appearances and positions of themoving object in two adjacent images are considered, and are perfectlyassociated according to the Bayesian decision, which is referred to asthe tracking of the moving object.

In detail, assuming at a time point t, a list containing n pedestriancandidate rectangles obtained based on object detecting and appearancemodel building is represented by C={P_(j) ^(t)|j=1, 2, . . . , n}, and ahistoric record tracked by a Bayesian tracker before a time point t−1 isa list M={H_(i) ^(t)|i=1, 2, . . . , m} that contains m visitorhypotheses. The so-called visitor hypotheses refer to the associated τcontinuous pedestrian candidate images H={P^(t−τ), P^(t−τ+1), . . . ,P^(t), ρ} under continuous tracking, wherein P^(t−τ) is a first appearedpedestrian candidate rectangle of the visitor, and others can be deducedby analogy. Moreover, ρ represents a confidence index, which isincreased or decreased corresponding to success or failure of the objecttracking, and when the confidence index is greater than an upperthreshold, the visitor hypothesis is considered to have enoughconfidence index, and is converted to a physical visitor; conversely, ifthe confidence index is less than zero, the moving object is regarded tohave left the monitoring scene, and now the visitor hypothesis can beremoved from the list M maintained by the Bayesian tracker. The aboveBayesian object tracking includes three stages of learning, associatingand renewing. In the following content, another embodiment is providedfor detailed description.

FIG. 5 is a flowchart illustrating a Bayesian object tracking methodaccording to an embodiment of the present invention. Referring to FIG.5, in the learning stage, a set of visitor hypothesis list M is firstprovided (step S510), wherein the visitor hypothesis list M includes aplurality of the visitor hypotheses that has been continuously trackedand associated.

Next, for each of the visitor hypotheses H_(i) ^(t−1) in the visitorhypothesis list M, whether a time length for the visitor hypothesisstaying in the image (time for being tracked) exceeds a firstpredetermined time L₁ (step S520) is detected, and if the time length isless than the first predetermined time L₁, the tracking is still in thelearning stage, and now the pedestrian candidates of the adjacent imagesare associated only according to a spatial correlation (step S530). Forexample, if the rectangular region b_(i) ^(t−1) belonged to the visitorhypothesis H_(i) ^(t−1) is spatially overlapped to the rectangularregion b_(j) _(t) of the pedestrian candidate P_(j) ^(t) of the currentimage, the visitor hypothesis H_(i) ^(t−1) is renewed to H_(i) ^(t) byadding the pedestrian candidate P_(j) ^(t).

Next, in the associating stage, i.e. the time length of the visitorhypothesis H_(i) ^(t−1) is greater than the first predetermined time L₁,it represents the visitor hypothesis is stably tracked, and now not onlythe spatial correlation is considered, but also the appearance featureof the object is considered to associate the visitor hypothesis to thepedestrian candidate according to the Bayesian decision (step S540). Indetail, such step includes calculating a spatial related priorprobability of the corresponding moving object in two adjacent imagesaccording to the above space information, calculating a likelihood ofthe corresponding moving object in the two adjacent images according tothe appearance information, and then combining the prior probability andthe likelihood to the Bayesian tracker to judge whether the visitorhypothesis is associated to the pedestrian candidate. For example, anequation (1) is a discriminate function of the Bayesian decision:

$\begin{matrix}\begin{matrix}{{B\; {D( {H_{i}^{t - 1},P_{j}^{t}} )}} = {{P( C_{H} \middle| P_{j}^{t} )}/{P( \overset{\_}{C_{H}} \middle| P_{j}^{t} )}}} \\{= {( {{p( C_{H} )}{p( P_{j}^{t} \middle| C_{H} )}} )/{p( {{p( \overset{\_}{C_{H}} )}{p( P_{j}^{t} \middle| ( \overset{\_}{C_{H}} ) )}} }}}\end{matrix} & (1)\end{matrix}$

Wherein, the likelihood function P(C_(H)|P_(j) ^(t)) represents theprobability of the pedestrian candidate P_(j) ^(t) belonging to thevisitor hypothesis H_(i) ^(t−1); the function P( C_(H) |P_(j) ^(t)) ison the contrary, i.e. represents the probability of pedestrian candidateP_(j) ^(t) not belonging to the visitor hypothesis H_(i) ^(t−1).Therefore, if BD is greater than 1, it represents that the decision isbiased in favor that the pedestrian candidate P_(j) ^(t) is belonged tothe visitor hypothesis H_(i) ^(t−1), and therefore the hypothesis andthe pedestrian candidate are associated. Wherein, if the similarityfunction p(P_(j) ^(t)|C_(H)) in the equation (1) is modeled by amulti-dimensional normal distribution N(μ,Σ²) an equation (2) isobtained:

$\begin{matrix}{{p( P_{j}^{t} \middle| C_{H} )} = {\frac{1}{\sqrt{\det \; {\sum( {2\; \pi^{d}} )}}}{\exp ( {{- \frac{1}{2}}( {f_{j}^{t} - \mu} )^{T}{\sum^{- 1}( {f_{j}^{t} - \mu} )}} )}}} & (2)\end{matrix}$

Wherein, μ and Σ are respectively a mean and a covariance matrix offormer L₁ feature values (from f_(i) ^(t−L) ¹ to f_(i) ^(t−1)), andcalculation methods thereof are as follows:

$\begin{matrix}{\mu = {\sum\limits_{k = {t - L_{1}}}^{t - 1}\frac{f^{k}}{L_{1}}}} & (3) \\{{\Sigma = \begin{pmatrix}\sigma_{11} & \sigma_{21} & \ldots & \sigma_{d\; 1} \\\sigma_{12} & \sigma_{22} & \ldots & \sigma_{d\; 2} \\\vdots & \vdots & ⋰ & \vdots \\\sigma_{1\; d} & \sigma_{21} & \ldots & \sigma_{dd}\end{pmatrix}}{{Wherein},}} & (4) \\{\sigma_{xy} = {( {f_{x} - \mu_{x}} )( {f_{y} - \mu_{y}} )}} & (5)\end{matrix}$

The likelihood function p(P_(j) ^(t)| C_(H) ) is represented by auniform distribution function. On the other hand, since the priorprobabilities p(C_(H)) and p( C_(H) ) are used for reflecting a priorrecognition of occurrence of the event, the prior knowledge iscorresponded to the spatial correlation. In other words, the closer adistance between the rectangular regions b_(i) ^(t−1) and b_(j) ^(t) is,the greater the prior probability is given. Here, an exponentialfunction related to the distance can be used for representing the priorprobabilities p(C_(H)) and p( C_(H) ), which are shown respectively asequations (6) and (7):

$\begin{matrix}{{p( C_{H} )} = {\exp( {- \frac{D( {b_{j}^{t},b_{i}^{t - 1}} )}{\sigma_{D}^{2}}} )}} & (6) \\{{p( \overset{\_}{C_{H}} )} = {1 - {p( C_{H} )}}} & (7)\end{matrix}$

Wherein, σ_(D) is a parameter controlled by a user, which can beadjusted according to a moving speed of the moving object in the image.Whether the visitor hypothesis is associated to the pedestrian candidateis judged according to the above spatial correlation and the appearancefeature (step S550). In the renewing stage, if the pedestrian candidateP_(j) ^(t) is judged to be associated to the visitor hypothesis H_(i)^(t−1), the pedestrian candidate P_(j) ^(t) is added to the visitorhypothesis H_(i) ^(t−1) to renew the visitor hypothesis into H_(i)^(t)={P_(i) ¹, P_(i) ², . . . , P_(i) ^(t−1), P_(j) ^(t), ρ_(i)} (stepS560). Meanwhile, a constant Δρ can also be added to improve theconfidence index ρ_(i) of the hypothesis until the confidence indexρ_(i) reaches a predetermined maximum value ρ_(max); conversely, if thevisitor hypothesis H_(i) ^(t−1) cannot be associated to any of thepedestrian candidate P_(j) ^(t) in the frame, the constant Δρ issubtracted from the confidence index ρ_(i), and when a value of theconfidence index ρ_(i) is less than zero, such visitor hypothesis isremoved from the visitor hypothesis list M, which represents that thevisitor has left the monitoring frame. On the other hand, if apedestrian candidate in the frame t cannot be associated to any of thevisitor hypothesis, it represents that the pedestrian candidate is anew-entering visitor, so that a new visitor hypothesis H_(m+1)^(t)={P^(t)} is added to the visitor hypothesis list M (Step S570), andρ_(m+1)=0 is given. In the present embodiment, the maximum value ρ_(max)is set to 1 and the constant Δρ is set to 0.1, though the presentinvention is not limited thereto.

To identify the appearance of the visitor entering and leaving the sceneto facilitate analysing a behaviour and a time point of a samepedestrian entering and leaving the scene, the present inventionprovides a visitor database to record the appearance model and the visittime of the visitor. Management of the visitor database is shown as aflowchart of FIG. 6, and detailed description thereof is as follows.

First, a new visitor hypothesis is added to the tracker (step S610), andwhether the time length of the visitor hypothesis reaches an integermultiple of a second predetermined time L₂ is judged (step S620), and ifyes, a mean feature vector and a covariance matrix thereof arecalculated according to L₂ former appearance features, so as to describea current appearance model V={N(μ,Σ),{s}} by the Gaussian function (stepS630), wherein {s} is a constant sequence recording a building time ofthe appearance model. Next, whether the time length of any visitorhypothesis equals the second predetermined time L₂ is judged (step S640)and whether the appearance model thereof is associated to the visitorappearance models recorded in the visitor database is also judged (stepS650). Wherein, if the visitor is similar to one or more visitorappearance models recorded in the visitor database (a distancetherebetween is less than a threshold T), it represents that the visitorhas ever visit the scene, and now the appearance model of the visitor isassociated to a most similar appearance model V_(k), and the visitorappearance model {tilde over (V)}_(k) in the visitor database is renewedaccording to equations (8) and (9) (step S660):

$\begin{matrix}{{\overset{\sim}{V}}_{k} = \{ {{N( {{\overset{\sim}{\mu}}_{k},{\overset{\sim}{\Sigma}}_{k}} )},\{ {s_{k}^{1},s_{k}^{2},\ldots \mspace{14mu},s_{k}^{u},s_{i}^{1},s_{i}^{2},\ldots \mspace{14mu},s_{i}^{v}} \}} \}} & (8) \\{{\overset{\sim}{\mu}}_{k} = \frac{{u \cdot \mu_{k}} + {v \cdot \mu_{i}}}{u + v}} & (9) \\{{{\overset{\sim}{\sigma}}_{k}^{2}( {x,y} )} = \frac{{u \cdot {\sigma_{k}^{2}( {x,y} )}} + {v \cdot {\sigma_{i}^{2}( {x,y} )}}}{u + v}} & (10)\end{matrix}$

Wherein, ρ²(x,y) represents elements (x,y) in the covariance matrix Σ, uand v are time sequence lengths of two appearance models, and served asweights for renewing. Since now V_(i) is a newly build appearance model,the v value thereof is 1. Conversely, if the new appearance model V_(i)cannot be associated to any appearance model within the visitordatabase, it represents that the visitor is a newly observed visitor,and now the appearance model and the time mark thereof can be added tothe visitor database (step S670). Now, a distance of the two appearancemodels (which are respectively a d-dimensional Gaussian distribution,N₁(μ₁,Σ₁) and N₂(μ₂,Σ₂)) is calculated according to the followingequations:

$\begin{matrix}{{{D( {{V( N_{1} )},{V( N_{2} )}} )} = {( {{D_{KL}( {N_{1}{}N_{2}} )} + {D_{KL}( {N_{2}{}N_{1}} )}} )/2}}{{Wherein},}} & (11) \\{{D_{KL}( {N_{1}{}N_{2}} )} = {\frac{1}{2}\begin{pmatrix}{{\ln ( \frac{\det \; \Sigma_{2}}{\det \; \Sigma_{1}} )} + {{tr}( {\Sigma_{2}^{- 1}\Sigma_{1}} )} +} \\{{( {\mu_{2} - \mu_{1}} )^{T}{\Sigma_{2}^{- 1}( {\mu_{2} - \mu_{1}} )}} - d}\end{pmatrix}}} & (12)\end{matrix}$

It should be noted that if the time length of the visitor hypothesis ismore than twice of the second predetermined time L₂, it represents thatthe visitor hypothesis has been associated to the visitor database, andnow the corresponding appearance model in the visitor database can becontinually renewed according to the equation (8).

Regarding the above threshold T used for judging the association of thetwo appearance models, if the distance between the two appearance modelsis greater than the threshold T, it can be judged that the twoappearance models respectively correspond to two different visitors;conversely, if the distance between the two appearance models is lessthan the threshold T, it can be judged that the two appearance modelsare associated, and accordingly it can be judged that the two appearancemodels correspond to the same visitor.

To calculate a most suitable threshold T, the present invention providesa unsupervised learning method, by which the system can automaticallyperform the learning and renewing according to the images, so as toobtain an optimal appearance distinguishing ability. According to themethod, two events A and B are considered. The event A represents that asame visitor is stably and continually tracked. As shown in FIG. 7( a)and FIG. 7( b), when the visitor is stably tracked by the system for thetime length of 2L₂, it has enough confidence to believe that the twoappearance models V₁′=AM(P^(t), P^(t−1), . . . , P^(t−L) ² ⁺¹) andV₁=AM(P^(t−L), P^(t−L) ² ⁻¹, . . . , P^(t−2L) ² ⁺¹) are come from thesame visitor, and now the distance D(V₁′,V₁) between the two appearancemodels can be calculated to serve as a feature value of the event A. Theevent B represents that two visitors are simultaneously appeared in themonitoring frame and are both stably tracked. As shown in FIG. 7( c),since the two visitors are simultaneously appeared in the samemonitoring frame, it has enough confidence to believe that the twoappearance models are come from different visitors, and now the distanceD(V₂,V₃) between the two appearance models can be calculated to serve asa feature value of the event B.

When an amount of the event A or the event B collected by the systemreaches a certain amount, a statistic analysis thereof is performed. Asshown in FIG. 8, since the feature value of the event A is obtained bycalculating the distances between the two appearance models built at twodifferent time points according to the same visitor, the feature valueof the event A is closed to a zero point and has a relativelyconcentrated distribution. The feature value of the event B is obtainedby calculating the distance between the two appearance models of twodifferent objects, so that the feature value of the event B isrelatively far away from the zero point and has a relatively loosedistribution. By respectively calculating a mean and a standarddeviation of the two events, and representing distance data thereof bythe normal distributions N_(A)(μ_(A),σ_(A) ²) and N_(B)(μ_(B),σ_(B) ²),a first distance distribution and a second distance distribution can bebuilt. Now, an optimal boundary of the first distance distribution andthe second distance distribution can be obtained according to anequation (13) to serve as the threshold T for distinguishing theappearance models:

$\begin{matrix}{{\frac{1}{\sigma_{A}\sqrt{2\; \pi}}^{{- \frac{1}{2}}{(\frac{\mu_{A} - T}{\sigma_{A}})}^{2}}} = {\frac{1}{\sigma_{B}\sqrt{2\; \pi}}^{{- \frac{1}{2}}{(\frac{\mu_{B} - T}{\sigma_{B}})}^{2}}}} & (13)\end{matrix}$

Finally, the appearance models and the elapsed time of the visitorsrecorded in the above visitor database are further applied to a wanderdetection, by which only the time sequence {s} recorded corresponding tothe appearance model of each of the visitors in the visitor database isrequired to be analysed. Here, the wander detection is performedaccording to the conditions of the following equations (14) and (15):

s _(t) −s ₁>α  (14)

s _(i) −s _(i−1)<β, 1<i≦t  (15)

Wherein, the equation (14) represents that a time for the visitorstaying in the frame from a first appearance to a current detection isgreater than a predetermined time α, and the equation (15) representsthat a time interval for the visitor being detected is at least lessthan a predetermined time β.

In summary, techniques of moving object tracking, visitor databasemanagement and adaptive threshold learning, etc. are combined in themethod for tracking the moving object of the present invention, by whichthe appearance model is built according to the appearance features ofthe moving object in a plurality of the images, and is compared to thedata stored in the visitor database that is configured in the systemmemory, so as to keep tracking the moving object. Even if the movingobject leaves the monitoring area and returns again, the visitor canstill be successfully associated to its former behaviour, so as toassist the supervisor in finding abnormal acts and making followingreactions.

It will be apparent to those skilled in the art that variousmodifications and variations can be made to the structure of the presentinvention without departing from the scope or spirit of the invention.In view of the foregoing, it is intended that the present inventioncover modifications and variations of this invention provided they fallwithin the scope of the following claims and their equivalents.

1. A method for tracking a moving object, comprising: detecting themoving object in a plurality of continuous images to obtain spaceinformation of the moving object in each of the images; capturing anappearance feature of the moving object in each of the images to buildan appearance model of the moving object; and combining the spaceinformation and the appearance model of the moving object to track amoving path of the moving object in the images.
 2. The method fortracking a moving object as claimed in claim 1, wherein the step ofdetecting the moving object in a plurality of continuous images furthercomprises: subtracting a background image from the images to detect themoving object.
 3. The method for tracking a moving object as claimed inclaim 1, wherein after the step of detecting the moving object in aplurality of continuous images, the method further comprises: marking aplurality of connection regions in the images; and estimating a minimumrectangular region capable of encircling the connection regions.
 4. Themethod for tracking a moving object as claimed in claim 3, wherein thesteps of detecting the moving object in a plurality of continuous imagesfurther comprise: judging whether the moving object is a trackingobject; and filtering the moving object not belonged to the trackingobject.
 5. The method for tracking a moving object as claimed in claim4, wherein the steps of judging whether the moving object is thetracking object comprise: judging whether an area of the rectangularregion is greater than a first predetermined value; and judging themoving object encircled by the rectangular region to be the trackingobject if the area is greater than the first predetermined value.
 6. Themethod for tracking a moving object as claimed in claim 4, wherein thesteps of judging whether the moving object is the tracking objectcomprise: judging whether an aspect ratio of the rectangular region isgreater than a second predetermined value; and judging the moving objectencircled by the rectangular region to be the tracking object if theaspect ratio is greater than the second predetermined value.
 7. Themethod for tracking a moving object as claimed in claim 3, wherein thesteps of capturing the appearance feature of the moving object in eachof the images to build the appearance model of the moving objectcomprise: dividing the rectangular region into a plurality of blocks,and capturing a color distribution of each of the blocks; obtaining amedian of the color distribution from each of the blocks according to arecursive method to build a binary tree for describing the colordistribution; and selecting the color distributions of branches of thebinary tree to serve as a feature vector of the appearance model of themoving object.
 8. The method for tracking a moving object as claimed inclaim 3, wherein the moving object is a pedestrian.
 9. The method fortracking a moving object as claimed in claim 8, wherein the step ofdividing the rectangular region into the blocks comprises dividing therectangular region into a head block, a body block and a lower limbblock according to a proportion of 2:4:4.
 10. The method for tracking amoving object as claimed in claim 9, wherein the step of capturing thecolor distribution of each of the blocks comprises neglecting the colordistribution of the head clock.
 11. The method for tracking a movingobject as claimed in claim 10, wherein the color distribution comprisesa color feature in a RGB color space or in a hue, saturation,illumination (HSI) color space.
 12. The method for tracking a movingobject as claimed in claim 10, wherein after the step of detecting themoving object in a plurality of continuous images to obtain the spaceinformation of the moving object in each of the images, the methodfurther comprising: tracking the moving path of the moving objectaccording to the space information, and accumulating a elapsed time ofthe moving object staying in the images.
 13. The method for tracking amoving object as claimed in claim 12, wherein after the step ofaccumulating the elapsed time of the moving object staying in theimages, the method further comprising: judging whether the elapsed timeof the moving object staying in the images is greater than a firstpredetermined time; and capturing the appearance features of the movingobject to built the appearance model of the moving object if the elapsedtime is greater than the first predetermined time, and combining thespace information and the appearance model of the moving object to trackthe moving path of the moving object in the images.
 14. The method fortracking a moving object as claimed in claim 13, wherein the steps ofcombining the space information and the appearance model of the movingobject to track the moving path of the moving object in the imagescomprise: calculating a spatial related prior probability of thecorresponding moving object in two adjacent images according to thespace information; calculating a likelihood of the corresponding movingobject in the two adjacent images according to the appearanceinformation; and combining the prior probability and the likelihood to aBayesian tracker to judge the moving path of the moving object in theadjacent images.
 15. The method for tracking a moving object as claimedin claim 13, wherein when the elapsed time is greater than the firstpredetermined time, the method further comprises: recording the elapsedtime and the appearance model of the moving object into a database. 16.The method for tracking a moving object as claimed in claim 15, whereinthe steps of recording the elapsed time and the appearance model of themoving object into the database comprise: associating the appearancemodel of the moving object to a plurality of appearance models withinthe database to judge whether the appearance model of the moving objectis recorded in the database; only recording the elapsed time of themoving object into the database if the appearance model of the movingobject is already recorded in the database; and recording the elapsedtime and the appearance model of the moving object into the database ifthe appearance model of the moving object is not recorded in thedatabase.
 17. The method for tracking a moving object as claimed inclaim 16, wherein the steps of associating the appearance model of themoving object to the appearance models within the database to judgewhether the appearance model of the moving object is recorded in thedatabase comprise: calculating a distance between the appearance modelof the moving object and each of the appearance models in the database,and judging whether the distance is less than a threshold; and renewingthe appearance model in the database according to the appearance modelof the moving object if the distance of the appearance model is lessthan the threshold.
 18. The method for tracking a moving object asclaimed in claim 17, wherein the step of renewing the appearance modelin the database according to the appearance model of the moving objectcomprises: selecting one of the appearance models with the distancebeing less than the threshold and most similar to the appearance modelof the moving object to renew the appearance model in the database. 19.The method for tracking a moving object as claimed in claim 17, whereinthe steps of calculating the distance between the appearance model ofthe moving object and the appearance models in the database comprise:calculating a first distance between the appearance models built at twodifferent time points based on the moving object in the images, so as tobuild a first distance distribution; calculating a second distancebetween the appearance models of two moving objects in the images, so asto build a second distance distribution; and obtaining a boundary of thefirst distance distribution and the second distance distribution toserve as a standard for distinguishing the appearance model.
 20. Themethod for tracking a moving object as claimed in claim 19, wherein thesteps of building the first distance distribution and the seconddistance distribution comprise: respectively calculating a mean and astandard deviation of the first distance distribution and the seconddistance distribution; and representing data of the first distancedistribution and the second distance distribution by a normaldistribution according to the mean and the standard deviation, so as tobuild the first distance distribution and the second distancedistribution.
 21. The method for tracking a moving object as claimed inclaim 15, wherein after the step of recording the elapsed time and theappearance model of the moving object into the database, the methodfurther comprises: analysing a time sequence of the moving object in thedatabase to determine whether the moving object is accorded to a wanderevent.
 22. The method for tracking a moving object as claimed in claim21, wherein the steps of analysing the time sequence of the movingobject in the database to determine whether the moving object isaccorded to a wander event comprise: judging whether a time for themoving object continuously appearing in the images is greater than asecond predetermined time; and judging the moving object to be accordedto the wander event if the time for the moving object continuouslyappearing in the images is greater than the second predetermined time.23. The method for tracking a moving object as claimed in claim 21,wherein the steps of analysing the time sequence of the moving object inthe database to determine whether the moving object is accorded to awander event comprise: judging whether a time interval for the movingobject leaving the images is less than a third predetermined time; andjudging the moving object to be accorded to the wander event if the timeinterval for the moving object leaving the images is less than the thirdpredetermined time.